ABSTRACT
Automated assessment of patients with Parkinson's disease (PD) is urgently required in clinical practice to improve the diagnostic efficiency and objectivity and to remotely monitor the motor disorder symptoms and general health of these patients, especially in view of the travel restrictions due to the recent coronavirus epidemic. Gait motor disorder is one of the critical manifestations of PD, and automated assessment of gait is vital to realize automated assessment of PD patients. To this end, we propose a novel two-stream spatial-temporal attention graph convolutional network (2s-ST-AGCN) for video assessment of PD gait motor disorder. Specifically, the skeleton sequence of human body is extracted from videos to construct spatial-temporal graphs of joints and bones, and a two-stream spatial-temporal graph convolutional network is then built to simultaneously model the static spatial information and dynamic temporal variations. The multi-scale spatial-temporal attention-aware mechanism is also designed to effectively extract the discriminative spatial-temporal features. The deep supervision strategy is then embedded to minimize classification errors, thereby guiding the weight update process of the hidden layer to promote significant discriminative features. Besides, two model-driven terms are integrated into this deep learning framework to strengthen multi-scale similarity in the deep supervision and realize sparsification of discriminative features. Extensive experiments on the clinical video dataset show that the proposed model exhibits good performance with an accuracy of 65.66% and an acceptable accuracy of 98.90%, which is much better than that of the existing sensor- and vision-based methods for Parkinsonian gait assessment. Thus, the proposed method is potentially useful for assessing PD gait motor disorder in clinical practice.
ABSTRACT
Motor disorder is a typical symptom of Parkinson's disease (PD). Neurologists assess the severity of PD motor symptoms using the clinical rating scale, i.e., MDS-UPDRS. However, this assessment method is time-consuming and easily affected by the perception difference of assessors. In the recent outbreak of coronavirus disease 2019, telemedicine for PD has become extremely urgent for clinical practice. To solve these problems, we developed an automated and objective assessment method of the leg agility task in the MDS-UPDRS using videos and a graph neural network. In this study, a sparse adaptive graph convolutional network (SA-GCN) was proposed to achieve fine-grained quantitative assessment of skeleton sequences extracted from videos. Specifically, the sparse adaptive graph convolutional unit with a prior knowledge constraint was proposed to perform adaptive spatial modeling of physical and logical dependency for skeleton sequences, thus achieving the sparse modeling of the discriminative spatial relationships. Subsequently, a temporal context module was introduced to construct the remote context dependency in the temporal dimension, hence determining the global changes of the task. A multi-domain attention learning module was also developed to integrate the static spatial features and dynamic temporal features, and then to emphasize the salient feature selection in the channel domain, thereby capturing the multi-domain fine-grained information. Finally, the evaluation results using a dataset with 148 patients and 870 samples confirmed the effectiveness and reliability of our scheme, and the method outperformed other related state-of-the-art methods. Our contactless method provides a new potential tool for automated PD assessment and telemedicine.